Developer Driven and User Driven Usability Evaluations

Software companies focusing on Usability Engineering facetwo major challenges, the first being the sheer lack ofusability specialists leading to missing competences in theindustry and the second, which regards small companiessuffering from the constraint of low budgets, thus not beingable to fund usability specialists or comprehensiveconsultancy. Training of non-usability personnel in criticalusability engineering methods has the potential of easingthese challenges. It is, however, unknown how much andwhat kind of research that has been committed to novicetraining in UE methods. This paper presents acomprehensive literature study of research conducted in thisarea, where 129 papers are analyzed in terms of researchfocus, empirical basis, types of training participants andtraining costs. Findings show a need for further empiricalresearch regarding long term effects of training, trainingcosts and training in user based evaluation methods. Author KeywordsTraining, developers, usability engineering, literaturereview. ACM Classification KeywordsH5.2 Information interfaces and presentation (e.g.,HCI):User Interfaces -User-centered design, Training, help anddocumentation. INTRODUCTIONDuring the last decade large software companies haveincreased their focus on introducing usability engineering(UE) methods into the development processes. Onechallenge for these companies is the sheer lack of usabilityspecialists in the industry, which leads to missingcompetences across the board and hence, problems withincorporating UE in the development processes [16, 24].However, small companies do not even have the privilegeof staffing usability specialists and these have to cope withissues such as the constraint of low budgets. In practice thismeans that small software companies do not have the fundsto pay for comprehensive consultancy or staffing ofusability specialists [13, 18, 28] as they are expensive tohire [23]. A survey conducted by Gulliksen et al.acknowledges this by showing that usability specialists areprimarily employed by medium-sized or large companies[12]. In their study 70 % of the 194 respondents, areemployed usability specialists in companies with at least 50employees, which are categorized as medium-sized [9].This is also supported by Rosenbaum et al. that shows thatmost usability specialists are employed in large companiesconsisting of 250 or more employees [25]. The fact that small companies usually do not have staff withusability competences is expressed as one of the mainbarriers for incorporating UE in the development process[22]. This is also supported in the survey presented in [25]that shows that 17.3 % of 134 respondents mentionedmissing competences as one of the main obstacles. One way of solving the problems of incorporating UE insoftware companies is to increase knowledge of the subjectacross the set of stakeholders [12]. Thus, training non-usability personnel in critical usability engineering methodshas the potential of easing problems regarding the lack ofusability specialists in the industry, which is experienced inlarge companies. Training also provides an opportunity forsmall companies to apply UE methods as the developersthemselves are driving the UE process, thus lessening theneed to staff usability specialists, which cannot be funded. Some studies provide promising insights regarding trainingof usability novices in UE methods. Metzker and Offergelddescribe a software project in which developers participatedin contextual task analysis, which motivated the participantsto produce components with a high level of usability [22].Another study presented in [17] describes how a non-usability specialist learned to apply a usability evaluationmethod. The participant stated that only little experiencewith cognitive psychology was needed in order to apply themethod. Seffah et al. describes the opposite case in whichusability specialists having sufficient technical knowhoweased integration of UE methods in the developmentprocess and gained wide acceptance from developers [29].Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires priorspecific permission and/or a fee.NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland.Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.